A server or computing system generally includes one or more processors, memory, and peripheral components and peripheral interfaces. Examples of peripheral components include cryptographic accelerators, graphics accelerators, and extensible markup language (XML) accelerators. Examples of peripheral interfaces include network interface cards (NICs), serial ATA (SATA) adapters, serial attached SCSI (SAS) adapters, RAID adapters, and Fiber Channel and iSCSI host bus adapters (HBAs). Processors, memory, and peripherals are often connected using one or more buses and bus bridges. To provide fault-tolerance, individual servers are often configured with redundant resources.
Since resources, such as peripheral components and peripheral interfaces, are assigned on a per server basis, other servers do not typically have access to these resources. In order to provide adequate resources for each server, resources are typically over-provisioned. For example, more hardware acceleration is provided than is typically needed. More network interface capacity is allocated than is typically used simply to handle worst-case or expected worst-case scenarios. Resources are over-provisioned resulting in overall waste and low utilization. Resource assignment on a per server basis also limits the ability to reconstruct or reconfigure a resource environment. A more efficient and flexible approach is to provide remote peripherals, such as storage, which can be shared among servers while maintaining quality-of-service guarantees and providing the ability to change dynamically the assignment of peripherals to servers. Such shared remote peripherals are referred to as virtualized resources.
Ethernet and InfiniBand are a commonly deployed server networking technologies and may be used for communication between servers and their remote peripherals. However, different reliability, performance, and quality-of-service guarantees are desirable for communication with remote peripherals. Consequently, additional adapter techniques and mechanisms are needed to provide efficient and reliable data transfer between servers in an InfiniBand or Ethernet network and remote peripherals.
For example, where a computer system has integrated disk storage, an interface such as the Small Computer System Interface (SCSI) can be used by the computer system to transfer data between the disk and memory. The integrated storage is typically only accessible to the computer system in which it is installed. A more efficient and flexible approach is to provide remote disk storage in a storage area network (SAN) which can be shared among servers while maintaining quality-of-service guarantees and providing the ability to change dynamically the assignment of peripherals to servers. In a Storage Area Network, the enterprise storage is maintained separate and apart from the computer system.
A fast networking technology called Fiber Channel can be used to transfer data between the enterprise storage and memory in the computer systems utilizing the enterprise storage. Fiber Channel connections utilize Fiber Channel Protocol (FOP) which is a transport protocol that transports SCSI commands over Fiber Channel networks. These connections are hot-pluggable and are usually implemented with optical fiber. A host bus adapter (HBA) connects a host computer system to other network and storage devices. In a particular HBAs can be used to connect a host computer system to remote data storage in a storage area network (SAN) in an enterprise data storage solution. A multi-channel host bus adapter card can be used to mediate data transfer between HBAs and FCP communication in a storage area network. This allows Fiber Channel to be used as a connection between an HBA of a computer server via the adapter to enterprise storage to transport SCSI information units, as defined by the Fiber Channel Protocol for SCSI (FCP). The multichannel host bus adapter card may be utilized as an I/O Director.
Data transfer with remote peripherals is described in U.S. patent application Ser. No. 11/200,761 entitled “Efficient Data Transfer Between Servers And Remote Peripherals” filed Aug. 9, 2005, which is incorporated herein by reference. An example of an I/O Director for connecting networked servers to a storage area network is described in U.S. patent application Ser. No. 12/544,744 entitled “Remote Shared Server Peripherals Over an Ethernet Network For Resource Virtualization” filed Aug. 20, 2009 which is incorporated herein by reference. An example of an I/O Director used in an InfiniBand network is described in U.S. Pat. No. 9,083,550 entitled “Network Virtualization Over InfiniBand” issued Jul. 14, 2015 which is incorporated herein by reference. These references describe some of the environments in which the invention of the present application, as described below, may be utilized.